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INTRODUCTION 

The  purposes  of  this  report  are  the  examination  of  existing 
techniques  of  classifying  a  system  of  linear  equations,    and,   if  the 
system  is  classified  as  ill-conditioned,   the  examination  of  iterative 
methods  available  by  use  of  which  it  is  possible  to  obtain  an  accur- 
ate representation  of  the  solution  of  the  system. 

A  system  of  equations  is  said  to  be  "ill-conditioned"  if  it  has  a 
solution  that  is  extremely  sensitive  to  slight  changes  in  the  coefficients 
of  the  variables.     Such  sensitivity  of  the  coefficients  impairs  the 
accuracy  and  dependability  of  the  solution  obtained  in  the  usual 
iterative  procedures,    especially  those  involving  matrix  techniques, 
because  of  round-off  error  which  is  inherent  in  these  techniques. 

Many  systems  of  equations  occur  as  a  result  of  experimental 
work.     The  coefficients  are  subject  to  error  resulting  from  measure- 
ment techniques,    etc.     Hence  it  is  necessary  that  the  stability  of  the 
system  be  known  so  that  appropriate  precautions  may  be  taken  to 
insure  a  reasonably  accurate  solution  of  the  system. 

The  solution  of  a  two-dimensional  system  is,    in  a  geometric 
sense,    represented  by  the  intersection  of  the  graphs  of  the  functions. 
As  illustration,    consider  the   set  of  linear  equations: 


4.  OOlx   +4.012x   =0.001       (a) 

1  *-< 


4.  012x  +4.  014x  =0.002.     (b) 

1  •— 


It  is  obvious  from  the  graph  that  a  slight  change  in  the  coefficients 
in  this  system  may  result  in  a  relatively  much  greater  change  in  the 
coordinates  of  the  point  of  intersection  of  the  lines. 


CLASSIFICATION  TECHNIQUES 


Consider  the  equations 


ailX  +  ai2y  =  dl 


a     x  +  a     v  —  d 
21  22y         2 


whereas  the  lines  corresponding  to  these  equations  are  nearly  parallel. 
The  angle(s)  between  the  lines  must  be  nearly  zero.     To  utilize  this 

fact,    one  classification  technique  is  the  consideration  of  the  angle 

2 
between  the  lines.     One  can  calculate  cos  0  or  cos    9  and  arbitrarily 

2 
decide  how  near  unity  must  cos  0  or  cos    0  be  to  term  the  system  of 

equations  ill-conditioned.     Stanton  (11)  infers  that  as  a  reference 

point  a  criterion  number  could  be  set  in  the  range  0.  90i.  05,    and  that 

2 
ill-conditionedness  may  result  if  cos    9  is  greater  than  this  number. 

This  would  make  9  slightly  less  than  18  degrees.     To  calculate  cos  9 

recall, 


|aHa21+ai2a22 
cos  9  =  .. where  9  is  the  angle 


x        2         2V2         2 
\ail+ai2     la21+a22 


between  the  lines  with  direction  numbers  a  ,  a  and  a  ,  a  . 
(Recall  that  these  are  also  the  coefficients  in  the  two  equations.  ) 
Generalized  to  n  dimensions  this  becomes: 


!  2 ; 


ik  jk 

cos  0..  = 


'  NJSaf^ 


2 

a.. 
Jk 


This  criterion  is  not  wholly  satisfactory  because  of  the 

2 
arbitrariness  involved  in  choosing  a  critical  value  for  cos    9. 

One  of  the  most  widely  used  methods  of  classifying  a  system  of 
equations  is  in  terms  of  the  value  of  the  determinant  of  coefficients. 
If  the  determinant  is  near  zero,    or  relatively  small  as  compared 
with  the  determinant  formed  by  replacing  the  coefficients  of  one  of 
the  variables  by  the  constants,   the  system  probably  is  ill-conditioned. 
Another  way  of  expressing  this  fact  involves  the  determinant  of  the 
normalized  coefficient  matrix.     If  the  absolute  value  of  this  deter- 
minant is  very  small  as  compared  with  unity  (i.e.    j  A     l«l)  then 
the  system  is  probably  ill-conditioned. 

Neither  of  the  two  preceding  methods  is  entirely  satisfactory  by 
itself.     As  illustration,    consider  the  set  of  linear  equations  already 
discussed: 

4. OOlx  +4. 012x  =0.001 

4.012x  +4.014x  =0.002. 


The  true  solution  of  this  system  is  x  =  -1  and  x   =  +1.      The  corres- 
ponding solution  obtained  by  method  of  elimination  with  round-off  in 


the  twelfth  position  is  x  =  -1.00000001819  and  x2  =  +1.00000001819. 
This  solution  agrees  quite  well  with  the  true  solution  considering  the 
fact  that  the  normalized  determinant  of  the  coefficient  matrix  is 
computed  to  be  0.  000705  which  is  quite  small  relative  to  unity. 

A.    M.    Turning  (12),    while  considering  the  solution  of  the  system 
AX=B,    suggests  that  instead  of  using  the  coefficient  matrix  A,   the 
matrix  A-S  be  used  where  S  is  a  small  predetermined  matrix  which 
he  calls  an  average  or  typical  matrix.     The  solution  of  this  new 
system  will  be  X  +A      SX     where  X     is  the  true  solution.     By  averaging 
the  effect  of  the  "average"  transformation  he  arrives  at  two  classifi- 
cation schemes,    one  termed  the  "N- condition  number  of  A"  and  the 
other  is  the  "M-condition  number  of  A.  "     The  "N-condition  number  is 
equal  to  (l/n)N(A)N(A~    )  where  n  is  the  dimension  of  A  and  N(A)  is 
the  norm  of  A.     The  second  condition  number  is  used  in  conjunction 
with  the  first.     The  "M-condition  number"  is  equal  to  n(M(A)M(A      )  ) 

where  n  is  again  the  dimension  of  A  and  M(A)  is  max  I  a..  J  .     The 

ij        1J 
coefficients  of  a  well- conditioned  matrix  give  the  "N-condition  number" 

of  order  n         and  the  "M-condition  number"  approximately  ln(n)  times 

larger.     The  larger  the  condition  numbers,    the  more  ill-conditioned 

the  system.     However,    as  in  the  other  classification  procedures,    this 

scheme  has  its  disadvantages.     For  one,    it  is  often  quite  tedious  to 

compute  A      .     For  another,    it  is  somewhat  arbitrary  as  to  how  large 


the  condition  numbers  need  be,    before  the  system  is  considered  ill- 
conditioned. 

Among  the  most  popular  measures  of  the  ill-conditionedness  of  a 
system  are  those  which  employ  knowledge  of  eigenvalues  and  eigen- 
vectors of  the  coefficient  matrix,    determination  of  which  in  itself  is 
often  a  challenging  problem.     The  usual  criterion  is  the  quotient 
I  f\  I    max.      The  larger  the  resulting  quotient,   the  more  ill-conditioned 

T?iTmin 

the  system  is.     The  disadvantage  of  this  method,    as  previously  men- 
tioned,   lies  in  the  computation  of  the  eigenvalues  and  eigenvectors. 

The  last  classification  scheme  being  considered  in  this  report  is 
attributed  to  Riley  (9).     When  attempting  to  solve  a  system  AX=B  where 
A  has  real  eigenvalues,   it  consists,    instead,    of  considering  the  solu- 
tion of  the  system  CY=B  where  C=A+kI.     The  matrix  C  is  "better 
conditioned"  than  is  the  matrix  A.     This  will  be  proved  by  showing  that 
if     \    ,      ^    ,    •    •    •    ,      A      are  the  eigenvalues  of  A  then    A    +k,      n?+k, 

*    *    *  .    n    +k  are  the  eigenvalues  of  C. 

To  show  that  this  is  so,   write  the  characteristic  equation  for  C: 

|rl-c|  =  0. 

It  follows,    successively,   that 


lrI-(A+kI)|  =  0, 
\rI-kI-Al'=  0, 
|(r-k)I-A[  =  0. 

If  the  eigenvalues  of  C  are    /\.,   then  the  eigenvalues  of  A  are    A.-k. 

Now,   without  loss  of  generality,    it  is  possible  to  assume  that  the 
eigenvalue  of  least  numerical  value  is  positive. 

Note,     n.+k^.  A.  where    A  .+k  and   fl  .  are  the  minimum  eigenvalues 

J  J  J  J 

of  C  and  A  respectively.     Thus  ^  .+k  ^  .  where    A.-    n.      is 

positive.     Hence,    it  follows,    successively,    that 


A.+k  n. 

J  '  J 

h.-   h-  a-    n 

_i L  l        J  <Q 


•'■  i  ^j         ,        ^i    > 

XTk".Yk       /\j+k       "   A) 

Xi  *    k      A^„ 


Aj+k      A.+k      A; 


Yk   -      ^    ■ 


Therefore 

lVkl    IM" 

Thus  the  matrix  C  is  better  conditioned  than  the  matrix  A. 

Riley  notes  that  his  experience  suggests  that  a  reasonable  choice 

2-s 

is  for  k,    a  small  positive  constant,   to  be  somewhere  between  10 

3-s 
and  10  where  s  is  the  number  of  decimals  being  carried.     X  is 

computed  from  a  series  expansion  in  a  matrix  Y  as  follows: 

C=A+kI  implies  A=C-kI.     Thus  one  has 

AA_1=(C-kI)A"1. 

Hence  it  follows  successively,   that 

I=CA_1-kA"  , 

C"1=C"1CA"1-C~1kA"   , 

A"1=C"1+kC"  A"  , 

A"1  =  C"1+kC"1(C"1+kC"1A"1)  =  C"1+kC"2+k2c"2A_1, 
Hence,  A_1  =  c"1+kC"2+  •    •    •    +knC_(n+1)+-    ■    •.     Thus 

X=A_1B  =  C"1B+kc"    B+  •    •    •+ knC"(n+1)B+  •    •    •    . 
Now,   letting  Y  =  C~    B,    it  follows  that 

X=Y+kC_1Y+  •    •    •    +(kC"1)nY+  •    ■    •    . 


To  obtain  an  indication  of  the  condition  of  the  system,   use  a  value 
of  k  of  one  in  the  next  to  last  decimal  place  (k=0.  000*    •    •  010).     Y  is 
then  computed  by  an  inversion  technique  such  as  the  Cholesky  or  Square 


Root  method.     Then  using  Y  as  the  right  member,    one  computes  the 
second  term  in  the  series  expansion  of  X;  this  is  then  used  to  obtain 
the  next  member  of  the  series.     If  it  appears  that  additional  compu- 
tation would  contribute  nothing,    it  is  likely  that  the  system  is  well- 
conditioned,    providing  k  was  appropriately  chosen.     On  the  other  hand, 
if  it  appears  that  additional  computation  would  contribute  to  the  sum 
of  the  series  then  the  system  is  ill-conditioned  and  thus  the  original 
system  must  have  been  quite  ill-conditioned.     The  disadvantages  of 
the  method  lie  in  the  choosing  of  suitable  k  and  in  the  series  computation 
of  X.     It  should  be  noted  that  in  order  for  the  condition  of  C  to  be  better 
than  that  of  A,    k  must  be  approximately  as  large  as  ]  A  .Imin.      However 
it  must  not  be  too  large,    lest  convergence  of  the  series  be  slowed  down. 
The  rate  of  convergence  depends  on  k  times  the  eigenvalues  of  C 

This  technique  presents  a  method  to  improve  an  approximate 
solution  of  the  system.     There  exist  matrices  A  such  that  A       cannot 
be  expressed  exactly  carrying  any  specified  number  of  decimals.     How- 
ever,  for  such  matrices  the  inverse  of  the  corresponding  C  can  be 
expressed  exactly  with  appropriate  choice  of  k  and  can  be  used  to 
improve  the  approximate  solution.     For  example: 

Let  X    be  an  approximate  solution  of  AX=B. 

Consider  "CZ   =B-AX  ". 
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Now  C(X-(X0+ZQ)  )  =  CX-CX0-CZ0 


=  AX+kX-AX0-kXQ-CZ0 
=  B-AX   -CZ   +k(X-XQ). 

Thus  X-(X0+Z0)=kC'1(X-XQ).     Now  letting  Xl=XQ+ZQ  and  Z^C"    (B-AXj), 

repeat  the  process.     Continuing,    one  notes  Z    =C      (B-AX   ).     Thus  if 

Z  — *  0,    one  has  X— *X   .     Convergence  of  Z     is  highly  dependent  upon  the 
n  n  n 

appropriate  choice  of  k,   as  is  seen  from  the  following  development. 
If  one  has 


CZ0=B-AX0, 
CZ  =B-AXlf 
CZ2=B-AX2, 
CZ3=B-AX3, 


CZ    =B-AX   , 
n  n 

then  Z.  =  C"1(B-AX.), 


0  v  0' 

Z1  =  C"1(B-AX1)  =  C"1(B-A(X0+Z())  ), 

Z2  =  C"1(B-AX2)  =  C_1(B-A(X1  +  Z1)  ), 

Z3  =  C"1(B-AX3)  =  C"1(B-A(X2fZ2)  ), 


Z    =C-1(B-AX   )  =  C_1(B-A(X      ,+Z      . )  ). 
n  n  n-1       n-1 


11 


Consider: 


Z3  =  C_1(B-A(X2+Z2)  ) 


=  z2-c 


=  z2-c" 
=  z2-c" 
=  z2-c" 

=  zz-c' 


AZ. 


A(ZrC 
AZ   +(C" 
AZ   +(C" 
AZ   +(C" 
AZ   +(C" 


AZX) 

A)2ZX 

A)2(Z0-C_1AZ0) 

A)2Z0-(C_1A)3Z0 

A)2Z()-(C"1A)3(C"1(B-AX0)  ). 


In  general,   for  Z    ,    one  has 
n 


Z    =Z      ,-C_1AZ      _  +  •    '    •+(-lC-1A)n"1Z.+(-lC"1A)n(C"1(B-AXA)  ). 
n      n-1  n-2  0  0 


No- 


*'•      L**G      C 


.£envalues  c 


—      —  T    ■    1    i    »     h    ■  _,     -  -  w     1 


1 


Thus  since  raising  C       to  the  nth  power  also  raises  the  eigenvalues  of 
C       to  the  nth  power,   the  eigenvalues  will  approach  zero.     Recall,    if 
the  eigenvalues  of  a  matrix  are  zero,    then  the  matrix  is  the  zero  matrix. 
Thus  an  appropriate  choice  of  k  which  will  allow  the  eigenvalues  of  C 
to  approach  zero  rapidly,   will  insure  convergence  of  the  series. 


MODIFICATION  OF  AN  ILL-CONDITIONED  MATRIX 


The  following  technique  is  one  which  presents  a  method  of  improving 
the  condition  of  the  coefficient  matrix  of  a  system  of  equations.     The 
procedure  is  concerned  with  the  eigenvalues  of  the  coefficient  matrix, 


12 


principally  those  which  are  very  small.     Thus,    since  most  methods 
for  extraction  of  eigenvalues  obtain  the  dominant  eigenvalue  first,   a 
technique  for  obtaining  the  smallest  eigenvalue  is  presented  in 
Appendix  A. 

As  illustration,    it  is  convenient  to  apply  the  method  to  a  four  by 
four  system  where  the  coefficient  matrix  is  real  and  symmetric.     Fur- 
ther,  for  the  present  it  is  assumed  that  the  eigenvalues  are  distinct 
and  are  such  that  |  Ai)>l  A  ->!  >l?wl  >l?V4i  '     A  method  attributed  to 
J.    W.    Head  and  G.    M.    Oulton  (2)  is  as  follows. 

Consider  the  ill-conditioned  system  AX=D  where 


^11     ai2    ai3    ai 


? 


with  a..=  a. .,         X= 


a  a  a  a 

21       22  23  24 

a31  a32  a33  a34 

a41  a42  a43  a44 


X 

and  D= 

'd|l 

y 

d2 

z 

d3 

t_ 

N 

The  equation 


f 


irAai2 


a  a    — 1 

13  14 


a21        a22' ^a23 


24 


u31 


k41 


32 


42 


a33-^a34 


'43 


44 


\ 


=  0 


is  satisfied  by  the  values  A,,    X?>    )\?>  ^4  which  allow  the  column 

vector  X  to  have  a  non-trival  solution.     If  to  the  non-trival  solution 

\  2      2      2     2 

sponding  to  A,  the  additional  condition  that  x  ^.0;  x  +y  +z   +t=l 


corresj 
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be  imposed,   the  resulting  X     =    Tx"^    will  be  a  normalized  eigenvector 


corresponding  to  A  ,.     This  vector  furnishes  the  "mode",    U   = 
xx.+yy.+zz   +tt   ,    associated  with   f\    .     In  a  like  manner  the  modes 
corresponding  to  the  other  eigenvalues  can  be  represented. 

Since  these  eigenvectors  X.  correspond  to  distinct  eigenvalues 
of  a  real  symmetric  matrix  they  are  orthogonal.     Hence  the  matrix 
P=  £x,   X?  X     X~J     is  normal,    orthogonal,    and  unitary.     Thus  these 

modes  satisfy  the  orthornormal  properties: 

,    .       2      2      2     2, 

1.  )    x   +y   +z   +t   =1 

r      r      r     r 

2.  )    x  x  +y  y  +z    z   +t  t   =0  for  all  r  £  s 

rsrs      rs     rs 

3-»  iv1-  &"1-  I.V1-  nt* 

4.  )    Vx   y    =     Vx    z    =       Vx  t    =        K"y    z    =    Cy   t    =    Cz    t   =0 
Z_i  r   r         Zj  r   r         <£j  r  r  ^L»   r    r        Z  /  r  r        J  «  r  r 


Then  the  solution  of  the  original  system  is: 

2 


\    x  ^    'x   y  \      'x   z 

">k  ">Br  *'& 


The  limit  of  summation  on  all  sums  ranges  from  one  to  four. 


x=d 


£v*  +  Sr-&  ♦  -.VI  -.V-^ 


t=diS 


X    t 

r  r 


'r 


*£a, 


d*LT  +  d* 


c-»z  t 


+  d 


r— I  2 


^  "It 
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To  show  that  this  is  so,    consider  the  original  system  AX-D. 

Let  X=    )   a      X     where  X     is  the  eigenvector  corresponding  to  A    • 
£j  r       r  r  r 


Also  let 


D=£br 


X      where  b.=X!D.     Thus  AX=D  becomes  A(  /    a   X   )  = 
r  j      j  *-*   r    r 


\  b   X   .     This  is  a  finite  sum,    hence  one  has  \a  AX  =  Vb   X   .     Thus 

\  a    X    X  =^\"Tb   X     which  implies  a.  =b./ A.. 
/,!    r  nr    r^rr  c  ill 


Hence 


\ 

\b  X       * 
>  r  r 

r^x  b 

\    r  r 

Xl  X2  X3  X4l 

Ax  hz   A3  X4J 

l 

b2 
b3 
\ 

Now  let 


\- 


^000 

o      }\  2     o       0 
o        o    X  3     o 


J 


0         0      ^ 


Then  X=    [X:    X2    X3    X^J  A  "*    ^    X£    X3    X^        D 


-1 


Thus 


~      Pxi      X2      X3      X? 

yl      y2      y3      y4 


Zl      Z2      Z3      Z4 
tl       t2      t3       *4 


£ 


J 


s  yi  zi  *r 

X2  y2  Z2  *2 

X3  y3  Z3  *3 

x4  ^4  z4  *4 


'dH 
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From  this  expansion  it  is  clear  that  large  coefficients  of  d 
are  contributed  mainly  by  terms  having  f\      in  the  denominator.     The 
terms  involving  this  eigenvalue  can  be  isolated  by  multiplying  the 
expression  for  x  by  x  ,   the  expression  for  y  by  y   ,   the  expression 
for  z  by  z    ,   the  expression  for  t  by  t     and  adding.     The  result  is  U4= 

xx4+yy4+zz4+tt4'    U4=(dlVd2y4+d3Z4+d4t4)/  ^  4'     Head  and  °Ult°n 
infer  that  all  uncertainty  in  the  solution  of  the  system  can  be  attributed 

to  the  fact  that  U     is  uncertain.     This  is  a  consequence  of  the  fact 
that  slight  errors  in  the  d.  are  greatly  magnified  when  the  expression 
is  divided  by  ^    . 

The  procedure  in  the  next  step  is  to  replace  one  of  the  equations 
of  the  original  system  by  the  expression  for  U    .     As  for  which  equa- 
tion to  replace,    it  is  suggested  that  if  the  left  side  of  any  equation  is 
approximately  proportional  to  the  expression  for  U  .,   then  that  equa- 
tion is  the  one  to  be  replaced.     If  no  equation  appears  proportional  to 
U     then  the  equation  to  replace  to  improve  the  condition  of  the  system 
is  arbitrary.     It  may  be  necessary  to  try  omitting  several  of  the 
original  equations  before  the  condition  of  the  system  is  improved.     The 
condition  will  be  improved  when  the  determinant  of  coefficients  of  the 
new  system  is  greater  than  that  of  the  original  system.     If,    in  the 
example  the  fourth  equation  is  replaced  by  xx   +yy   +zz   +tt   =U   ,    the 
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determinant  of  the  coefficient  matrix  will  be  /\    f[  ^  fy  _  I  x 


X2      X3 


*1      ^2      ^3 


Zl      ZZ      Z3l 


To  show  that  this  expression  is  the  determinant  of  the  new  system, 

let  B=Tanl      a,,      a,,      a,"\        .     Consider 

11         12        13         14 

a21      a22      a23      a24 

a31      a32      a33      a34 

4         '4  4  4  _, 


the  matrix  product  B  £x      X      X      xl     Note,    since^X      X^   X3    X4"J 
is  unitary,    its  determinant  is  one. 


Now 


BV 


11         12         13         14 


a21      a22      a23      a24 


a31      a32      a33      a34 


LX4 


a,  1    fx 


V  z  t 

y4  4  4 


Thus 


r 


TanV^zV^V*!^! 


a21Xl+a22>rl+a23Zl+a24tl 
a31Xl+a32yl+a33Zl+a34tl 


BX 


i=  K 


X 


: 


A  similar  procedure 
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confirms  that      BX 


,=  *,r 


and  BX. 


-y  r 


However,    BX   = 


railVai2Vai3Z4+ai4t4 
a21X4+aZ2y4+aZ3Z4+a24t4 
a3lVa32y4+a33Z4+a34t4 

[X4X4    +y4y4    +Z4Z4    +V4=1_ 


Hence  if  the  determinant  of  B  \X     X      X      X~]    is  expanded  in  terms  of 
the  elements  of  the  fourth  row  the  indicated  expression  is  obtained. 
Similar  expressions  are  obtained  when  one  of  the  other  equations  is 
replaced  by  the  expression  for  U  ..     These  determinants  have  the  pro- 
perty that 


Xl    yl    Zl 

X2   y2    Z2 
lx3  y3  z3 


Xl  yl  *1| 
X2  y2  *2| 
X3   y3   *3 


+ 


Xl    Zl    *1 
X2    Z2   *2 


x3    z3    .3 


s 


yl  zl  *ll 

y2    Z2   *2l 
y3    Z3   H 


=  1 


since  P  is  orthogonal  and  unitary.     Thus  not  all  of  the  determinants 

can  be  small.     Consequently,   the  condition  of  the  system  will  be  improved 

when  the  equation  which  corresponds  to  one  of  these  minors  is  replaced 

byU4. 

It  should  be  noted  that  if  additional  data  cannot  be  obtained  to  fix 


the  value  of  U     very  precisely,    then  useful  information  can  be  gained 
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with  U   =0. 
4 

If  there  exist  two  small  eigen  values  then  the  homogeneous 
system  of  equations  from  which  the  eigenvectors  are  derived  reduces 
to  two  equations  with  four  unknowns.     Thus  a  non-trival  solution  must 

be  obtained  in  terms  of  two  of  the  unknowns.      The  procedure  is  to 

2      2      2     2 

find  the  non-trival  solution  with  the  condition  x  +y  +z   +t   =1  and  the 

additional  condition  that  one  of  the  unknowns  be  zero.     Let  this  solution 

be  denoted  as  x  ,   y   ,    z    ,    and  t    .     Then  another  non-trival  solution 

2      2      2     2 
is  obtained  with  the  condition  x  +y   +z   +t  =1  and  xx  +yy  +zz   +tt=0. 

These  two  non-trival  solutions  yield  the  corresponding  "modes": 
U3=xx3+yy3+zz3+tt3=(x3d1+y3d2+z3d3+t3d4)/  ^  3 

U4=xx4+yy4+zz4+tt4=(x4di+y4d2+z4d3+t4d4)/  ^     . 

The  procedure  now  is  the  same  as  that  for  the  distinct  eigenvalue  case 
with  the  exception  that  two  equations  of  the  original  system  are  replaced 
by  the  expressions  for  U     and  U    . 

After  employing  the  preceding  technique  for  improving  the 
condition  of  the  coefficient  matrix  one  may  proceed  to  solve  the  resulting 
system  of  linear  equations  by  the  same  method  used  to  solve  a  system 
of  well- conditioned  equations. 
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AN  ITERATIVE  TECHNIQUE 

This  technique  employs  a  modification  of  the  coefficient  matrix 
similar  to  that  used  by  Riley  in  his  classification  technique.     The  pro- 
cedure is,   when  contemplating  the  solution  of  the  ill-conditioned  system 
AX=D,   to  consider  instead  the  system  (A+K)X=D+KX  where  K=kl  and  k 
is  a  small  positive  constant.     These  two  systems  are  mathematically 
equivalent,   but  the  matrix  A+K  is  better  conditioned,   as  was  previously 
shown. 

The  method  is  to  delete  the  term  KX  and  solve  the  resulting 
system  (A+K)Y=D.     The  solution  is  denoted  as  Y       .     Thus  if  X  is  the 
true  solution  then  X-Y       =E  is  the  error  after  the  first  iteration. 

Now  if  one  subtracts  (A+K)Y=D  from  AX=D,   the  result  is  AX-(A+K)Y=0. 
Thus  A(X-Y)=KY.     Therefore,   AE^     -KY(     .     Hence  one  has  obtained  a 
new  system  with  the  same  coefficient  matrix  A.     Modifying  the  coefficient 
matrix  in  the  manner  above  one  then  solves  the  system  (A+K)Y       =KY 

Then  the  error  after  the  second  iteration  is  given  by  E       =E       -Y  "  = 

(1)       (2)  .  ^(m)   ^(m-1)    „(m)    „    "«r^  „(i) 

X-YV     -Yv    '.     After  m  iterations,    Ev     '=EV  '-Y*       -X-  j>_j  Yv  '. 

i=l 

Thus  X=^r*  Y^'+E*       .     The  formula  for  the  determination  of  the  Y  J 
i=l 

is  (A+K)Y(j)=D  if  j=l 

=KY(J_1)  ifj>l. 
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If  the  method  converges,    the  solution  of  the  original  system  is    X- 
To  obtain  conditions  for  convergence  of  the  series,    consider: 


Y(1^(A+K)_1D 


Y^A+KfW™-1). 


Therefore,   let  (A+K)_1D=C  and  (A+K)~    K=B.     Thus 


y(m)=BY(m-l) 

Y(m-l)=BY(m-2) 


Y(3LBy(2) 
Y(2)=BY(D  . 


„(m)    „m-l_ 
Hence  Yv     '=B  C.     Also 


2yu,=(i+b+bV  •  •+Bm-i)c 


If    2    bJ  converges  then  (A+K)X=D+KX  implies 


j  =  0 


X=(A+K)"1D+(A+K)"1KX 


=  C+BX. 


«=*o 


lc=  2 


j, 


Thus  X-BX=C  and  X=(I-B) 

3=1 
m  oo        ,    m-1 

Hence,    X-  ]T   Y0)=  (  2    fiJ- "S    bJ)C 
j=l  j=0  j=0 


3  =  0 

=  2    BJC=(I-B)"iB111C 


j_    ,_    „,-l„m, 
jJ  r*— i 


m 


^"^/T  „,-l^,  -^(m) 

=B     (I-B)      C=EV     ' 


cc 


,(m). 


Thus  if  S^     BJ  converges  then    lim      B     =0.     Therefore  E^     '=0  and 
J=0 

oo 
X=S     Y{j)  . 

j=0 

CO 
Since  the  choice  of  k,   which  will  insure  the  convergence  of  ^  ]  B    , 

(m)  j  =  ° 

is  a  difficult  problem  and  then,   the  computation  of  E  can  become 

tedious,    consider  an  expression  for  the  upper  bound  of  error.     This 
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expression  is  developed  as  follows, 

-1 


(m)-^^™-1) 


Define  (A+K)"    =D=  [d.1    .     Recall  Ylm'=DKY 


Thus  Y 


(m)  _ 


(m) 

1 

(m) 


(m) 


n       r 


d  d      ■ 

11         12 


d  d      • 

21         22 


d  d      • 

nl         n2 


'J 


2n 


nn 


wm-iJ\ 


ky 


1 
(m-1) 


ky 


(m-1) 
n 


__  (m)  (m-1),.      .     (m-1)  ,      ,     (m-1) 

Hence,    yv         =d    .  ky,  '+d    _ky;  '+•    •    • +d      kyv  ':r=l,..n 

r  rl       1  r2    '  2  rn   ;n 


Thus  one  has 
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|y(rm^hri<-1,|+K2^m'1,l+---  +  ldrn<m-1,| 


**WMY~ 


(m-1) 


(m-1)  (m-1) 

where  y  =  max  y. 

'm  1 


Since  this  holds  for  all  r,    it  will  hold  if  |y^       I      is  replaced  by 

i^HMri  •  •  •  ••  Ki}- 


Therefore 


y  I  :  r=  1,    2,    .  .  .  n. 


(c) 


Let  d,   =max 

M 


(SIM-  2M  ••••Bd4  • 

Then  define  a  convergence  constant  H  and  restrict  H  so  that  |k|  d     =H<  1 
This  restriction  can  always  be  satisfied  by  choosing  k  sufficiently  small 
From  (c),   for  rn>l,    one  has 


(m) 


y 


M 


/m-1) 
m 


1^  H     provided  y(m_1^  0. 


m 


(d) 


Hence  the  series 


CO 

s: 

m=l 


(m) 

M 


is  absolutely  convergent  since  H<1, 


Considering  the  expressions  for  the  individual  components  of 
the  matrix  X  one  observes 


m 


co 


,(j> 


0  j  =  l  j  =  m+l 
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Thus  the  error  after  m  iterations  for  each  x.  is 

1 

m  co 

e.(m)=x.-2     y.(j)=    TJ    y.(j):i=l,2 n. 

1  m+1 

Also    |e.(m))£  YL    jy^^^l.Z,...,   n. 

Now  since    ly.        }<£  jy^      j     ,     from  (d)  one  obtains 


H 


This  technique  is  highly  adaptable  to  machine  computation  as 
the  coefficient  matrix  used  when  obtaining  a  solution  of  the  modified 
system  is  the  same  for  each  of  the  Y      .     It  is  presented  here  merely 
as  a  supplement  to  the  usual  methods  for  solution  of  systems  of 
equations. 
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APPENDIX  A 

In  the  determination  of  small  eigenvalues  only  those  which  are  less 
than  0.  2  in  absolute  value  are  being  considered.     This  is  so  since  if  the 
eigenvalue  exceeds  0.  2,   then  the  maximum  contribution  possible  to  any- 
coefficient  is  numerically  less  than  5.  0. 

In  this  procedure  it  is  assumed  that  the  evaluation  of  a  determinant 
or  the  solution  of  a  polynomial  in       with  numerical  coefficients  presents 
no  great  problem. 

Since  attention  is  focused  on  those  values  of  less  than  0.  2,   the 
technique  is  to  evaluate  the  determinant  of  the  characteristic  matrix  with 

=  -0.2+0.4r/n,    r  =  0,    1,    .  .  .  n,   where       is  possibly  negative.     The 
evaluation  of  these  determinants  gives  a  rough  idea  of  the  number  and 
position  of  the  roots. 

To  illustrate  the  procedure,    consider  the  following  characteristic 


matrix: 

-» 

y 

2.557- 

A 

2.624 

2.468 

2.624 

3.493-J\ 

2.351 

2.468 

2.351 

2.557-/\ 

2.351 

1.532 

2.624 

2. 

361 

- 

1. 

532 

2. 

624 

3. 

493- 

*J 

Since  all  values  of  (i  are  positive  for  this  example,   the  determinant, 
A(  ?\)»    is  evaluated  with  A  =0,  J\  =0.  04,  ^\  =0.  08,  7\  =0.  12,  }\  =0.  16. 
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The  corresponding  values  for  Am)  are 

A(0)  =  0.05 

A(0.  04)  =  0.  0019526 
A(0.08)  =  0.01761476 
A(0.  12)  =  0.  09101456 
A(0.  16)  =  0.2190776. 

Now /\  (  {[)  for  a  system  of  four  equations  is  a  polynomial  of 
degree  four  in  j\  .     Thus  f(  (\  ),    defined  by 

*i\\.    A/m  (A  -0.  04)(  A-0.  08)(  /\-0.  12)(A  -0.  16) 

H(\)-L\W)  0>04         0>08         012>  0l6 

-  A(0.04)    /N(/S-0.08)(A-0.12)(^-0.16) 

0.04     0.04  0.08  0.12 

+    A(0-08)    ?\(^-0.04)(  A-0-12)(/\-0.l6) 
0.08     0.04  0.04  0.08 

-  A(0.12)    >\(  /\-0.04)(  ?\-0.08)()\-0.  16) 

0.12     0.08  0.04  0.04 

+  A(°-16)    /V  /N-0.  04)(>\-0.  08)( A-0.  12) 
0.16     0.12  0.08  0.04 

is  identical  to^(^)  when  A  has  these  five  values.     Hence  f{/\)  will 
represent  j\  (  (\)  very  accurately  for  small  values  of  f\. 

Thus 


26 

f(  A)  =  0.  05  (?\4-0.4  /\3  +  0.056^2-0.  0032^+0.00006144 


0.00006144 

0. 

,0019526  (/V-0. 

36^3  +  0.04l6?l2-0. 

001536/^) 

0.00001536 

+ 

0, 

0171476  (/V-°- 

32/\3  +  0.0304  }\    -0. 

000768?!) 

0.00001024 

_ 

0, 

,09101456  (/\4-( 

).  28  ?\3+0.0224  ?\2-( 

).000512^) 

0.00001536 

+ 

0, 

,2190776  (A4-0. 

24  /\3+0.0176  AZ-0. 

,000384/\)  . 

0.00006144 

This  simplifies  to 

£{}[)  =  9449.49115?i4  -  835.  28266  A3  +  74.  19092^ 
-  3.8285^  +  0.05. 

The  technique  at  this  point  is  to  equate  £{f\)  to  zero  and  solve 
by  an  iterative  measure  for  f\  . 

In  this  particular  example  the  value  of  f\  obtained  is  0.  05. 
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The  purposes  of  this  report  are  to  define  "111- Conditioned 
Equations,  "  and  examine  existing  methods  of  classifying  a  system 
of  equations,    develop  a  procedure  to  improve  the  condition  of  the 
system  of  equations,    and  present  an  iterative  technique  for  use  in 
the  solution  of  a  system  of  "111- Conditioned  Equations.  " 

A  set  of  "111- Conditioned  Equations"  is  a  system  of  equations 
which  has  a  solution  that  is  extremely  sensitive  to  slight  changes  in 
the  coefficients  of  the  variables.     As  a  result,    the  accuracy  and 
dependability  of  the  solution  obtained  in  the  usual  iterative  procedures 
is  impaired. 

The  classification  techniques  considered  are: 

1.  The  (generalized)  angle(s)  between  the  "lines"  which 
represent  the  system  of  equations  in  a  geometric  sense. 

2.  The  ratio  of  the  determinant  of  the  coefficient  matrix  to 
the  determinant  formed  from  it  by  replacing  the  coefficient 
of  one  of  the  variables  by  the  constant. 

3.  The  consideration  of  the  determinant  of  the  normalized 
coefficient  matrix  as  compared  with  unity. 

4.  The  use  of  "M-condition  numbers"  and  "N-condition 
numbers.  " 

5.  The  largest  ratio  of  the  absolute  values  of  two  eigenvalues. 


6.     A  consideration  of  the  convergence  of  a  series  expansion 
of  the   system  of  equations. 

A  procedure  developed  to  improve  the  condition  of  the  original 
system  is  to  replace  one  of  the  original  equations  with  a  "mode.  " 
These  modes  are  derived  from  eigenvalues  and  their  corresponding 
eigenvectors.     A  technique  for  determination  of  eigenvalues  less  in 
absolute  value  than  0.  2  is  presented  in  Appendix  A. 

The  remainder  of  the  report  deals  with  an  iterative  technique 
which  is  highly  adaptable  to  electronic  computation.     An  expression 
for  the  determination  of  the  error  after  m  iterations  is  also  presented. 


